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Abstract. The determination of scalars involved in Lusztig's conjecture con- 
cerning the characters of finite reductive groups was achieved by Waldspurger 
in the case of finite classical groups Sp2n (Pg) or On(Fq) when p, q are large 
enough. Here p is the characteristic of the finite field Fq. In this paper, we 
determine the scalars in the case of Sp2n{Pq) with p = 2, by applying the 
theory of symmetric spaces over a finite field due to Kawanaka and Lusztig. 
We also obtain a weaker result for S02n{P q) with p = 2, of split type. 



0. Introduction 

Let G be a connected reductive group defined over a finite field Fg of cliarac- 
teristic p with Frobenius map F . Lusztig's conjecture asserts that, under a suitable 
parametrization, almost characters of the finite reductive group coincide with 
the characteristic functions of character sheaves of G up to scalar. Once Lusztig's 
conjecture is settled, and the scalars involved there are determined, one obtains a 
uniform algorithm of computing irreducible characters of G^ . Lusztig's conjecture 
was solved in [SI] in the case where the center of G is connected. In [S2], the scalars 
in question were determined in the case where G is a classical group with connected 
center, when p is odd, and the scalars are related to the unipotent characters of 
G^ . By extending the method there, Waldspurger [W] proved Lusztig's conjec- 
ture (or its appropriate generalization) for Sp2n and 0„ assuming that p, q are 
large enough. He also determined the scalars involved in the conjecture. But these 
methods cannot be applied to the case of classical groups with even characteristic. 

In this paper we take up the problem of determining the scalars in the case of 
classical groups with p — 2. We show that the scalars are determined explicitly in 
the case where G Sp2n with p = 2. We also obtain a somewhat weaker result 
for the case S02n of split type, when p = 2, containing the case related to the 
unipotent characters. The main ingredient for the proof is the theory of symmetric 
spaces over finite fields due to Kawanaka [K] and Lusztig [L4] . They determined the 
multiplicity of irreducible representations of G occurring in the induced module 

qF^ 

Ind-QF 1 in the case where the center of G is connected (for arbitrary characteristic). 
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Using this, one can determine the scalars for G in many cases for a connected 
classical group with connected center, with arbitrary characteristic. 

On the other hand, it was shown in [S3] that there exists a good representatives 
of for a unipotent class C in 5^271 or SOn for arbitrary characteristic. This 
implies that the generalized Green functions of turn out to be polynomials in 
q (more precisely, rational functions in g if p = 2) for various extension field Fgm , 
and so certain values of almost characters are also rational functions in q. This 
makes it possible to apply some sort of specialization argument for the character 
values of G^ for any m > 1, and one can determine the scalars of G^ which are 
related to the unipotent characters, from the result for G^ . Thus we rediscover 
the results in [S2]. But this method works also for p = 2, and from this we can 
deduce the result for G^ . 

The main result of this paper was announced in the 4th International Confer- 
ence on Representation Theory, Lhasa, 2007. 

1. Lusztig's conjecture 

1.1. Let k be an algebraic closure of a finite field of characteristic p. Let G 
be a connected reductive algebraic group defined over k. We fix a Borel subgroup 
B of G, and a maximal torus T contained in B, and a Weyl group W = Ng{T)/T 
of G with respect to T. Let VG be the bounded derived category of constructible 
Q;-sheaves on G, and let A^G be the full subcategory of VG consisting of perverse 
sheaves. Let S{T) be the set of isomorphism classes of tame local systems on T, i.e., 
the local systems C of rank 1 such that ~ for some integer n > 1, invertible 
in k. Take a local system C G S{T) such that w* C ~ C for some w G W . Then 
one can construct a complex E VG as in [L2, III, 12.1]. For each £ G S{T) we 
denote by Gc the set of isomorphism classes of irreducible perverse sheaves A on G 
such that A is a constituent of the i-th perverse cohomology sheaf pH''{K^) of 

for any i, w. The set G of character sheaves on G is defined as G = \JceS{T) 

1.2. We consider the Fg-structure of G, and assume that G is defined over Fg 
with Frobenius map F. We assume that B and T are both f -stable. We assume 
further that the center of G is connected. Let G* be the dual group of G and T* a 
maximal torus of G* dual to T. By fixing an isomorphism t : fc* ~ Q'/^ (Q' is the 
subring of Q consisting of elements whose numerator is invertible in fc), we have an 
isomorphism / : T* ~ S{T) (see e.g., [SI, II, 1.4, 3.1]). Let W* = Ng{T*)/T* be 
the Weyl group of G*. Then W* may be identified with W = Nq{T)/T, compatible 
with /. F acts naturally on S{T), via F~^ : C ^ F*C, and the action of F on 
<S(T) corresponds to the action of F~^ on T* via /. 

For each s G T* such that the conjugacy class {s} in G* is F-stable, we put 

Ws = {wgW*\ w{s) = s}, 
Z, = {weW*\ F{s) = w{s)}. 

Then Zg is non-empty, and one can write Zs = z-^Wg for some element z\ G Z^. 

Since the center of G is connected, Zq* (s) is connected reductive, and Wg is a Weyl 
group of Zg'{s). We choose z\ so that 7 = 7^ = : ^ Wg leaves invariant 

the set of simple roots of Zg* (s) determined naturally from B and T. 
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Similarly, for any L G 5(T) such that F* L ~ £, we define 
Wc = {w^W\ w*C ~ £}, 
Zc = {w&W\ F*C ~ [w-^fC]. 
Then Wc (resp. Zc) is naturally identified with Wg (resp. Zg). 

1.3. Let IrrG^ be the set of irreducible characters of . Then IrrG^ is 
partitioned into a disjoint union of subsets £{G^ , where s T* and {s} runs 
over all the F-stable semisimple classes in G* . According to [LI], two parameter 
sets X(Ws,7) S'lid X{Ws,j) are attached to £{G^ ,{s}), and a non-degenerate 
pairing { , } : X{Ws,j) x X{Ws,j) Qi is defined. Here X{Ws,"/) is a finite set, 
and X{Wa,j) is an infinite set with a free action of the group M of all roots of 
unity in Q*. More precisely, there exists a set X{Ws) with 7-action, and a natural 
map X(V1^,,7) X{Ws) whose image coincides with X{Ws)'^, the set of 7-fixed 
points in X{Ws). In the case where 7 acts trivially on Ws, X{Ws, 7) coincides with 
X(Ws,7) X M. In general, the orbits set X{Ws,'y)/M is in bijection with X{Ws)'^. 

Now the set £{G^ , {s}) is parametrized by X{Ws,'j)- We denote by py the 
irreducible character in £{G^,{s}) corresponding to y G X{Ws,j). In turn, for 
each X G X{Ws,'y), an almost character is defined as 

(1.3.1) i?, = (-i)'(^^) Yl 

where A(y) = ±1 is a certain adjustment in the case of exceptional groups Ey, Eg. 
If c is the order of 7 on Wg, the almost characters R^ are determined, up to a c-th 
root unity multiple, by the M-orbit of x in X{Ws,'y). 

1.4. It is known by [L2, V], that the set Gc is parametrized by X{Wc) = 

X{Ws) imder the identification Wc — Ws- For each y G X{Ws), we denote by Ay 
the corresponding character sheaf in Gc- Let G^ be the set of F-stable character 
sheaves, i.e., the set of A G G such that F*A ~ A. Then G^ = U^^^, where 
C runs over the elements in S{T) such that [Fw)* C ~ C for some w € W . The 
set is parametrized by X(Ws)'''- For each A G G^, we fix an isomorphism 
: F*A ^ A as in [L2, V, 25.1]. Then (j)A is unique up to a root of unity multiple. 
We define a class function xa = XA,<j>A the characteristic function G^ Qi 
of A. In the case of classical groups, we have the following theorem, which is a 
(partial) solution to the Lusztig' conjecture. 

Theorem 1.5 ([SI, II, Theorem 3.2]). Assume that G is a (connected) classical 
group with connected center. Then for each x G X{Ws,^), there exists an algebraic 
number Cx of absolute value 1 such that 

Rx = CxXAsi J 

where x is the image of x under the map X{Ws,^) — > X{Ws)'^ . 

1.6. Assume that G is a connected classical group with connected center. Let 
P be an F-stable parabolic subgroup of G containing B, and L be an F-stable Levi 
subgroup of P containing T, Up the unipotent radical of P. Then Wl = Nl{T)/T 
is a Weyl subgroup of W, and Bl = B Ci L is a. Borel subgroup of L containing T. 
Let L be the set of character sheaves on L. We assume that Lc contains a cuspidal 
character sheaf Aq for C G <S(T), where C is Fw-stable for some w G W. Then Aq 
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may be expressed by the intersection cohomology complex as Aq — IC{S, £) [dim E] , 
where E is the inverse image of a conjugacy class in G = G/Z^{G) under the natural 
map TT : G ^ G, and £ is a cuspidal local system on S. The pair or its 

restriction on the conjugacy class, is called a cuspidal pair on G. Then either L 
is the maximal torus or L has the same type as G, and Aq is a unique cuspidal 

— • • • G 

character sheaf contained in Lc- Consider the induced complex K = indp on 
G. Then K is a semisimple perverse sheaf on G whose components are contained 
in G. By Lemma 5.9 in [SI, I], the endomorphism algebra EndMG K is isomorphic 
to the group algebra Qi[Wf] of Ws, where 

We = {n€ Ng{L) \ nEn'^ = E,ad{n)*£ ~ 

Z£ = {n& Ng{L) I F{nEn-'') = E, {Fn)*£ ~ £}/L. 

On the other hand, if we choose a positive integer r large enough, the set 
£{L^ , {s}) contains a unique cuspidal character i5 of , where s G T* corre- 
sponds to jC under /. We define 

Ws = {we Nw{Wl) I wBlw-'^ = Bl,"" S ^ S}, 
Zs = {wG Nw{Wl) I wBlw-'^ = Bl,^"" 5 ~ 6}. 

Since Lc contains a unique cuspidal character sheaf, we have Wg — We, Zg — Zs 
by [SI, I, (5.16.1)]. Moreover there exists wi G Zs such that Zg = wiWg and that 
71 = Fwi : Ws — > Ws gives rise to an automorphism of the Coxeter group Wg. 
We denote by (Wg)^ the set of irreducible characters of Wg, and (Wg)^^ the subset 
of (Wg)^ consisting of 71-stable characters. Then ^0 is Fwi-stable, and for each 
E G (Wg)^^, there exists Xe G X{Ws,^) and xe G X{Ws)'' such that Ae = A^^ is 
an F-stable character sheaf in Gc which is a simple component of K corresponding 
to E G Endx K. Moreover, p^^ € £{G^ , {s}) is an F-stable irreducible character 

which is a constituent of the Harish-Chandra induction IndpF"- S corresponding to 
E G (Wg)^^, and the image of the Shintani descent Shpr-fF of Pxs determines the 
almost character R^e of G^. (For the Shintani descent, see [SI]). 

1.7. Let Lyj^ be an F-stable Levi subgroup twisted by F{wi), i.e., L^i = 

aLa~^ for a G G such that a~^F{a) = F{wi) for a representative wi G Ng{L) 
of wi G Zs. Then by ad(a~^) : L^^ I^i, ad(a~^)*^o gives rise to an F-stable 
cuspidal character sheaf on Lyj^ which we denote by Aq. If we fix an isomorphism 
(fio '■ {Fwi)*£ 2^ £, cpo induces an isomorphism (p™^ : F*Aq 2:^ Aq on Lyj^ . We choose 
(t>A'„ ■■ ~ as (j)A'^ = iPo^- Then by Theorem 1.5, we have 

(1-7.1) Ro""' = CoXA', 

for some Co G of absolute value 1. where Rq""^ is the almost character of L^^ 
corresponding to Aq G L^-^. The following result was proved in [SI] in the course of 
the proof of the main theorem. (Note that in [SI], the constants Eq^Ao and SqS^Ae 
are used. But the proof shows that these constants are indeed given by Co = eoCao-) 

Lemma 1.8 ([SI, II, Lemma 3.7]). Let Co be as in (1.7.1). Then we have 

= (-i)^™''Coxa. 

for any E G {Wg)^^. 
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1.9. Lemma 1.8 shows that the determination of the scalars C,x appeared in 
Theorem 1.5 is reduced to the case of cuspidal character sheaves. We note that 
it is further reduced to the case of adjoint groups. In fact, let Aq be an F-stable 
cuspidal character sheaf contained in Gc- Let tt : G — ^ G be as before. Then 
Ao can be written as ~ 5o ® TT*Ao[d\, where d = dimZ(G), and So is a local 
system on G which is the inverse image of £q e iS(G/Gdor) under the natural map 
G — > Gder (Gder is the derived subgroup of G), and Aq is a cuspidal character 
sheaf on G. Since Aq is a unique cuspidal character sheaf in Gc, Aq is F-stable. 
Then Tr*Ao is _F-stablc and so £o is also F-stable. Then (pAo '■ F*Ao Oi^Aq is given 
by = fo ® ''■*0Ao' where (p^o '■ F*Ao ^^Aq is the map chosen for Aq, and (fo 
is the pull-back of the canonical isomorphism F*£'q!::^£'q. Hence xao is written 
as XAa = Oq (g) tt*Xao' where t^*XAo pull-back of XAo under the induced 

map TT : G^ G^ , and is a linear character of G^ corresponding to £o- A 
similar description works also for almost characters. Let Rq (resp. Rq) be the 
almost character of G^ (resp. G^) corresponding to Aq (rcsp. Aq). Then we have 
Rq = Oo CSi Tr*Ro. (This follows from the fact that if (5 is a cuspidal irreducible 
character of G^ corresponding to Aq for sufficiently large r, then 5 can be written 
a,s S = 9(S)S, where ^ is a cuspidal irreducible character of G^ corresponding to Aq, 
and is an F-stable linear character of G^ , and by applying the Shintani descent 
on 6.) Thus Co for Aq coincides with Co for Aq. 

2. generalized Green functions 

2.1. Under the setting in 1.6, we further assume that G^ is of split type. 
Let L be as before. Assume that Aq is a cuspidal character sheaf on L of the form 
Aq = IC(Z', f )[dim Z"], where E = Z^{L) x G with a unipotent class C in L and 
£ = QliM£' for a cuspidal local system £' on G. Then Wf = W = Ng{L)/L. 
For each w e W, let be an J'-stable Levi subgroup of G obtained from L by 
twisting w as in 1.7, i.e., = aLa~^ with a € G such that a~^F{a) = F{w) 
for a representative w € Ng{L) of w. Put = aSa~^,£w = &d{a~^)*£, a local 
system on Ey^. We assume that the pair (G, £') is i^-stable, and fix an isomorphism 
tpo : F*£' 2:^£'. Then one can construct an isomorphism ((po)u) : F*£w2^£w as in 
[L2, II, 10.6], and this induces an isomorphism ip^ : F* K^, where is a 
complex induced from the pair [E^, £w)- Note that is isomorphic to indp Aq, 
with a specific mixed structure twisted by w G W. We denote by XK^,ip^ the 
characteristic function of with respect to i^^. 

Since L is of the same type as G, and F is of split type, 71 : W ^ W is identity. 
Let K = indp Aq — Q^^y^A Ve Ae be the decomposition of K into simple 
components, where Ae is a character sheaf corresponding to E E W^, and Ve is 
the multiplicity space of Ae which has a natural structure of irreducible W-module 
corresponding to E. Then there exists a unique isomorphism (^Ae ■ F*Ae2:^Ae 
for each E G such that 

(2-1-1) XK„,^„= Yl '^{w,Ve)xae- 

Let Guni be the unipotent variety of G, and G^^^; be the set of i^-fixed points in Guni- 
The restriction ofxK„,ip„ on G^^^ is the generalized Green function Q^^ £, (^^^y 
([L2, II, 8.3]), where G^ = aCa~^,£^ = a,d{a~^)*£', local system on G^, and 
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((/5o)^ is the restriction of {(po)w on £[^. On the other hand, by the generahzed 
Springer correspondence, for each E S W^, there exists a pair (Ci,£i), where Ci 
is a unipotent class in G and £i is a G-equivariant simple local system on Ci, such 
that 

(2.1.2) ^ij|G„„i ^lC{Ci,£i)[dimCi+ dim Z"{L)]. 

Now the pair {Ci,£i) is F-stable, and (p^E '■ F*Ae !^Ae determines an isomor- 
phism -tps^ : F*£i2i,£i via (2.1.2) (cf. [L2, V, 24.2]). In other words, the choice 
of ipo : F*£' 2i£' determines tp£^. Let X(Ci,£i) • C'f Q; be the characteristic 
function of £i with respect to Lusztig ([L2, V, 24]) gave an algorithm of com- 
puting xAe = XAE,<j>AE '^^ ^uni- Here XAeIg" expressed in terms of a linear 
combination of various X(C(,f()) where (C(, £"{) is a pair as above corresponding to 
some E' e W^. Let pe,e' be the coefficient of X(C[,e[) in the expansion of xae- 
Then ps^^' satisfies the following property; if we replace F by for any integer 
m > 0, we obtain a similar coefficient pe,e' (with respect to ) starting from 
t/jg"*'' : {F"^)*£' 2^£' induced naturally from <y9o, which we denote by p^^pEi- Then 

there exists a rational function Pe,e'{x) such that p^^, = Pe,e'{q"^)- (Note : It 
is shown in [L2, V] that Pe,e' turns out to be a polynomial if p is good.) 

Now X(Ci.£i) is described as follows; take u € Cf and put Ag{u) = Zg(u)/Zq{u). 
Then F acts naturally on Ag{u), and in our setting F acts trivially on it. The set 
of G^-conjugacy classes in is in 1:1 correspondence with the set Ag{u) (note: 
Ag{u) is abelian). We denote by a„ a representative of a G^'^-class in Gf corre- 
sponding to a e Ag{u). On the other hand, the set of G-equivariant simple local 
systems on Gi is in 1:1 correspondence with the set Ag{u)^ of irreducible characters 
of Ag{u). For each p e Ag{u)^, we defnie a function fp on G^^; by 

if V = Ua € Gf , 

if f ^ Gf 

for V e Gfj^;. Let p S Aq{u)^ be the character corresponding to £i. Then there 
exists rjE & Q;* of absolute value 1 such that 

(2.1.4) XiCuSi) = VeJp- 

Note that rjE depends on the choice of ipo : F*£' 2^ £' and on the choice of u e Gf . 
We have the following theorem. 

Theorem 2.2 ([S3]). LetG be a classical group, simple modulo center. Assume 
that the derived subgroup of G does not contain the Spin group. Further assume 
that G is of split type. Then for each unipotent class Gi in G, there exists u e Gf 
(called a split unipotent element) satisfying the following; Let (C,£') be the pair in 
L as in 2.1 and uq G be a split element. Choose ipo ■ F*£' !^£' so that the 
isomorphism ((/?o)iio ■ ^uo ~^ ^uo induced on the stalk £^^ of £' at uq is identity. 
Choose a split element u G Gf for defining fp in (2.1.3). Then r]E = ^ for any 
E G W^. 

2.3. Returning to the setting in 2.1, we choose a split element u G Gf for 
each unipotent class Gi of G. Then u S Gf for any integer m > 0, (in fact, it is a 
split element with respect to G^""), and we choose ui™^ e Gf", a representative of 
G^ -class in Gf for each a G Ag{u). For later discussion, we prepare a notation. 



(2.1.3) fp{v) 
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Assume given a family of functions h — {/i^'"^}m>o, where /i^™^ is a class function 
on . Then we say that /i is a rational function in q if there exists a rational 
function Hci,a{x) for each pair (Gi,a) such that = HcA^f)- 

For each E G VV^, wc have an isomorphism cj)-]^^ : {F"^)* Ae I^^b, and one 
can define a function = Xa ("<■) on . Then in view of Theorem 2.2, 

Lusztig's algorithm implies that {xAeIc^"^ }m>o is a rational function in q. 
Thus, by (2.1.1) we have the following corollary. 

Corollary 2.4. Assume that G is as in Theorem 2.2, and that G^ is 0/ split 
type. The generalized Green function ^^^^ can he expressed as a rational 

function in q. 

2.5. More generally, if there exists a family of values h — {/i^"*^ g Qi}m>o 
such that /i^"*) = for some rational function H{x), we say that his a rational 

function in q. 

3. Cuspidal character sheaves 

3.1. Let G be an adjoint simple group of classical type. Wc assume that 
G is of split type over Fg. Let G° be the set of cuspidal character sheaves on G. 
A £ G^ is given in the form A = IC(G,f )[dimG], where G is a conjugacy class 
in G, and £^ is a simple G-equivariant local system on G. We shall describe the 
cuspidal character sheaves on G (cf. [L2, V, 22.2, 23.2], see also [S2, 6.6]) and their 
mixed structures. 

(a) G = PSp2n {n > I) with p : odd. G^ is empty if n is even. Assume that 
n is odd. Then for each pair {Ni,N2) such that Ni = df + di for some integers 
di > and that n = Ni + N2, one can associate cuspidal character sheaves on G 
as follows. Let G be a conjugacy class of g = su = us, where s is a semisimple 
clement of G such that Zq{s) is isomorphic to H — {Sp2Ni x SP2N2)/ and u 
is a unipotent element of Zq{s) ~ H such that the unipotent class Go containing 
u gives a unique cuspidal pair (Go, ^o) with unipotent support of H. Here (Go, ^o) 
is doHcribcd as follows. There exists a cuspidal pair (Ci,£i) for Sp2Ni such that 
Go = Gi X G2 and £"0 = ^1 ^ £2- Choose u = (ui,U2) G Go such that Ui € Gj. 
Let pi G AHi{ui)'^ corresponding to £i, where Hi = Sp2Ni- Then pi lEl p2 € 
{Ahi{ui) X Ah2{u2))^ factors through Ah{u) and defines an irreducible character 
po of Ah{u) corresponding to £0. 

Now assume that Ni ^ A^2- Then Zg{s) is connected, and so Acig) = Ah{u), 
and pq gives an irreducible character p £ Acig)'^ which determines a local system 
£ on G, and we denote by Ani,N2 the character sheaf corresponding to {C,£). 
Next assume that A''i = A^2- Then Ag{s) Z/2Z and Ah{u) is a subgroup of 
Aa^g) of index 2. We have Ind^^^^^-'^ po = p + p', where p, p' are linear characters of 
Aq(u). If we write £, £' the simple local system on G corresponding to p, p' , then 
the pairs (G,f ), (G,f ) are both cuspidal pairs of G. We denote by Aat^^jv^, 
the cuspidal character sheaves on G corresponding to (G, £),(G, f ), respectively. 
The set G° consists of these elements. 

We shall fix a mixed structure on (G, f). Since s G G^, H is i^-stable, and 
so (Go,5o) is also F-stable. Choose u = (ui,U2) € Gq such that u, are split 
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elements in Sp2Ni, and fix s e appropriately. Wc choose cpo : F*£ so that 
the induced isomorphism {fo)g '■ ~^ on the stalk £g at g is identity. Then 
Lpo induces an isomorphism tp : F*^7Vi,Ar2 ^^^1,^2- We define (pA = iPani.nz 
(j)A = qr(d™G-dimC)/2^_ ^ similar construction is apphed also for (C, f ). 

(b) G = PSOm {fn > 3) with p : odd. is empty unless m is either odd or 
divisible by 8. Note that PSOm = SOm if m is odd. To each pair {Ni,N2) such 
that Ni = d1 for some dj > 1 and that m = A^i + A^2, one can associate cuspidal 
character sheaves A associated to (C, £) as follows. Let C be the conjugacy class of 
G containing g = su = us, where s is a semisimple element such that H = Zq{s) 
is isomorphic to SOn^ x SOn2 if fn is odd, and to {SOni x 5*0^^2 )/{±1} if m is 
even, and « is a unipotent element in Zq{s) ~ H such that the unipotent class Co 
containing u gives a unique cuspidal pair {Go,£o) with unipotent support on H. 
Here Co = Ci x C2, £0 o± £1^ £2 with the cuspidal pair {Ci,£i) on S'Oat.. Choose 
u = {ui,U2) G Co such that Ui ^ Ci. Let pi G A/f. (uj)^ corresponding to £i, where 
Hi = SONi- Then pi Kl p2 G (ylHi(ui) x A/f2(^^2))^ factors through Aniu) and 
gives an irreducible character po g Ah{u)^ corresponding to £0. Depending on the 
structure of Aa{s), the three cases occur. 

(i) The case where Ni = or N2 = 0. In this case, Zg{s) is connected and so 
Ag{u) ~ Ah{u). Po gives p € which determines a local system £ on C, 
and iC,£) corresponds to the cuspidal character sheaf A7Vi,Ar2- 

(ii) The case where A^i > 0, > 0, iVi ^ N2. Then Ag{s) ~ Z/2Z, and 
Ah{u) is regarded as an index 2 subgroup of Acig)- We have Ind^^^^'^j po = p + p' 
for p, p' e ^0(5)^- If we write £, £' the simple local system corresponding to p, p' , 
{C,£),{C,£') arc both cuspidal pairs for G. We denote by AjVi.ATs) ^jvi JV2 
cuspidal character sheaves corresponding to them. 

(iii) The case where Ni = N2. In this case Ag{s) ~ Z/2Z x Z/2Z and so 
AG{g)/AH{u) ~ Z/2Z X Z/2Z. Ind^^^^^^^ po decomposes into 4 irreducible (lin- 
ear) characters, p, p' , p" , p'" of AG{g)- Correspondingly, we have simple local sys- 
tems £,£',£",£"' on C, and all of them give cuspidal pairs on G. We denote by 
^jVi,Ar2)^jVi JV2' ^JVi ATa' '^Ni N2 cuspidal character sheaves corresponding to 
them. 

All of the above three cases give the set G°. We shall fix a mixed structure 
on cuspial character sheaves. Since s G G^ , H is F-stable, and so {Co,£o) is F- 
stable. Take u = (mi,U2) G Cq such that Ui are split elements in SOni, and fix 
s e T^. We choose ipo : F*£ £ so that the induced isomorphism {(fo)g '■ ^g ^ ^g 
on the stalk £g at g is identity, (fo induces an isomorphism (p : F*A 2^ A for 
A = ^Wi,JV2- We define 0a by 0a = g(diniG-dimC)/2^_ -y^g ^ggj^g similarly for 
4' 4" A'" 

(c) C = 5p2n > 1) with p = 2. G° is empty unless n = c?^ + d for some 
d > 1. Assume that n = (P + d. Then G contains a unique cuspidal pair (C, £). The 
set G° consists of a single character sheaf A associated to {C,£). We fix a mixed 
structure of A. C is an F-stable unipotent class of G, and we take a split element 
u € G^ . We fix an isomorphism Lpo : F*£ 2^£ so that the induced isomorphism 
{(fo)u ■ £u ^ on the stalk f„ of £ at zt is identity, (po induces (f : F*A2iA. We 
define 0a by 0a = g(dimG-dimC)/2^ 
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(d) G = S02n with p = 2 (n > 1). G° is empty unless n = AcP for some 
d> 1. Assume that n = Acf. Then G contains a unique cuspidal pair (C, f). The 
set G° consists of a single character sheaf A associated to {C,£). C is an F-stable 
unipotent class of G, and we take a split element u G . We define (pA in a similar 
way as in the case (c), by = g(d™G-dimC)/2^_ 

We show the following lemma. 

Lemma 3.2. Let p he the irreducible character of Ac{g) corresponding to the 
local system £ on C, as in 3.1. Then p is a linear character such that p^ = 1. A 
similar fact holds also for p',p",p"' if there exists any. 

Proof. Let p be one of the characters p, p' , p" , p"' if there exists any. It is 
enough to show that pia"^) = 1 for any a G Acig). By investigating the structure 
of we sec that AG{g) is an elementary abelian 2-group if Ni ^ ^2- Thus 

in this case, p(a^) = 1. Wc assume that iVi = N^- Then p is odd, and G is PSp2n 
or PS02n- Assume that G = PSp2n- Wc have Aaig) — (c) ^Ah{u), where a is 
an element of order 2 permuting two factors of Ah{u). In this case a G Acig) is of 
order 2 or 4. If a has order 2, there is nothing to prove. Assume that a has order 
4. Then we have a? G Ah{u). Put 9 = Ind^^|^'!j po- Since Ah{u) is an elementary 
abelian 2-group, and po is a-stable, we see that 0{a'^) = \AG{g)\/\AH{u)\. This 
shows that p(a^) = 1 for any irreducible factor p of 9. Next assume that G = S02n- 
In this case, ^^(g) ~ (a) xAniu), where Ah{u) is an elementary abelian 2-group 
containing Ah{u) as an index 2 subgroup, and a is an element of order 2 acting on 
Ah{u). <j stabilizes Ah{u) permuting their two factors, and po is u-stable. Thus a 
similar argument shows that p(a^) = 1. The lemma is proved. □ 



4. Symbols and unipotent characters 

4.1. Irreducible characters contained in £(G^ , {1}) arc called unipotent char- 
acters. In the case of classical groups, unipotent characters are parametrized by a 
combinatorial object called symbols. In this section, we review unipotent characters 
of classical groups. 

Let G be a classical group over Fg of type Bn,Gn or _D„. We assume that 
G^ is of split type if G is of type £>„. The set of unipotent characters of G^ is 
parameterized by symbols. A symbol is an (unordered) pair (y) of finite subsets 
of {0, 1,2,...} modulo the shift operation (^) - {^',) with 5" = {0} U {S + 1), 
T' = {0} U (T + 1). The rank of a symbol A = (|) is defined by 

where [z] denotes the largest integer which does not exceed z. The defect d{A) 
of A is defined by the absolute value of IS*] — |T|. The rank and the defect are 
independent of the shift operation. 

For each integer d > 0, we denote by the set of symbols of rank n and defect 
d. In the case where A = (|!) is defect 0, A is said to be degenerate if S = T, and is 
said to be non-degenerate otherwise. We denote by $° the set of symbols of rank 
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n and defect 0, where the degenerate symbols are counted twice. We put 

d:odd d=0 (mod 4) 

Then the unipotent characters of of type Bn or C„ (resp. Dn of spht type) 
are parametrized by $„ (resp. $^). In the notation of 1.3, Wg ~ W and 7 = 1 
since F is of spht type, and $„ or <i>+ is nothing but X{Ws,j) = X{W,1). We 
denote by pA the unipotent character of G^ corresponding to A G or . The 
unipotent cuspidal character exists if and only if n = (P+d (resp. n = Ad"^) for some 
integer (i > 1 if G is of type Bn or G„ (resp. I?„). In these cases, the symbol Ac 
(the cuspidal symbol) corresponding to the (unique) cuspidal unipotent character 
is given as follows. 

Ac = (^0' 1' 2- • • ' g ^2<i+i : type B„ or G„, n = d' + d), 



0,l,2,...,4d- 1 



e<^tf {G: type Dn,n = 4d'). 



A = Am = 



4.2. We introduce a notion of families in or Two symbols A, A' belong 

to the same faniilv if A. A' are represented by {^) , {^',) such that SUT = S' UT' 
and that S CiT = S' CiT' . Families give a partition of or <!>+. A symbol yl £ 
of defect 1 is called a special symbol if ^ = (^) with S = {ao, ai, . . . , &,„} and 
T = {bi, . . . , bm} such that ao < 61 < oi < ■ • ■ < 6m < o-m- Similarly, a symbol 
A G $+ of defect is called a special symbol if yl = (^) with S = {ai, . . . ,am}, 
T = {61, . . . , bm} such that ai < 61 < • • • < < bm- Each family contains a 
unique special symbol. Let .F be a (non-degenerate) family. Then any symbol 
A £ can be expressed as 

Z2UM 

for some M, where Z\^Zi are determined by T\ Zi is the set of elements which 
appear in both rows of /I, Z\ is the set of singles in A, and M is a subset of Z\. 
The map M 1— > Am gives a bijective correspondence between the set of subsets M 
of Zi such that |M| = rfi (mod 2) and where = 2di + 1 (resp. \Zx\ = 2di) 
ii T C $„ (resp. c $^) for some integer di > 1. (In the case oi T C <I>^, we 
further assume that the smallest element in M is bigger that that of Zi — M.) In 
particular, the special symbol in can be written as Amq for some Mq c Zi such 
that |Mo| = di. For M G Zi, put M* = Mo U M - Mo n M. We define a pairing 
{ , } : .F X ^ ^ Q by 

(4.2.1) {^M,ylM'} = ^(-l)l^""^'"l 

where / = di (resp. / = di — 1) if C (resp. C <I'^). We extend this pairing 
to the pairing on or by requiring that F" and F' are orthogonal \i T ^ T' , 
which we denote by the same symbol. Note that the pairing { , } on or 
coincides with the pairing { , } on X(Wi 1) given in 1.3. Hence, for each A £ T, 
the almost character Ra is given as 

(4.2.2) Ra= K^'W' 
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where X„ = $„ or $+ according to the cases G is of type Bn or C„, or G is of type 
Dn- By the property of the pairing { , }, one can also write, for each A£ J^, 

(4.2.3) PA= {A,A'}Ra'. 

4.3. Assume that contains a cuspidal unipotent character, and denote 
by Tc the family containing the cuspidal symbol Ac- Then the special symbol Aq 
contained in Tc is given as follows. 

" (i°3^' ^' ' 2d -'^l) '■ Br,OTCn,n = £ + d), 

/0,2, . . . ,4(i- 2\ ,^ ^ , ,2^ 

In the case where G is of type Bn or G„, we have Zi = {0, 1, ... , 2d} and Mq = 
{1,3,..., 2c?— 1}. In the case where G is of type Dn, we have Zi = {0, 1, . . . ,4d— 1} 
and Mo = {1, 3, . . . , 4(i — 1}. In both cases, Ac is given by Ac = Am with M = 0. 
We denote by Rq = Ra^ the cuspidal almost character. 

4.4. Let Aq be the cuspidal character sheaf of G corresponding to i?o = Ra^- 
Let (G, f) bo the cuspidal pair corresponding to Aq. Then it is contained in the 
list in 3.1. If j3 = 2, it is uniquely determined since G° consits of a single element. 
In the case where p ^ 2, the explicit correspondence is known by [S2, Prop. 6.7], 
see also [L3]. The conjugacy class is given as follows (though we don't need it 
in later discussions). We use the notation in 3.1. Let g = su = us € G with 
H = Zq{s). Assume that G = PSp2n with n = rf^ + rf. Then H is isogeneous 
to Spd2+d X Spd2+d- Assume that G = PS02n+i with n = + d. Then H is 
isogenous to 50(^+1)2 x SOcp. Assume that G = PS02n with n = 4d^. Then H is 
isogeneous to 80442 x 80442 . 

5. Symmetric space over finite fields 

In this section, we apply the theory of symmetric space over finite fields to the 
problem of determining the scalars Cx occurring in Lusztig's conjecture (Theorem 
1.5)] for G^\ 

5.1. Let G be a connected reductive group over a finite field with Frobenius 
map F. Wc consider the symmetric space G^ /G^ . For a class function / on G^ , 
we define TO2(/) by 

m2(/) =(Indg:' 1,/)g.2 = E /(^)- 

In the case where G has a connected center, m2(p) is determined by Kawanaka [K], 
Lusztig [L4] for any irreducible character p of G^ . 

5.2. Let G be an i^-stable conjugacy class in G. Take x G and let Aa{x) 
be the component group of Zg{x) as before. F acts naturally on Ag{x). We assume 
that F acts trivially on Ag{x). Then the set of G^-conjugacy classes in G^ is in 
bijection with the set Aa{x)/^ of conjugacy classes in Ag{x). The correspondence 
is given as follows; for each a S Ag{x), take a representative d G Zg{x). There 
exists ha & G such that h~^F{ha) = d. Then Xa = haxh~^ in contained in G^, 
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and the set {xa \ a G Ac{x)/ ~} gives a complete set of representatives of the 
(j-'^-conjugacy classes in . 

The above description works also for the case of . We denote by {?/„ | a e 
Ag{x)/^} the set of -conjugacy classes in . We define a class function 
on for each r e Ag{x)^ as follows. 



(5.2.1) fr{g) = 




if g is G-'^ -conjugate to ya, 

We have the following lemma. 

Lemma 5.3. Let t e Ag{x)^ be a linear character such that = 1. Then we 
have 

m2ifr)^\C^\/\G^\. 

Proof. Take g G . Then g is G^-conjugate to an Xa & G^ for some 
a G Ag{x). Then there exists h € G such that g = hxh~^ and that h~^F{h) = a G 
^^(a;). Since F acts trivially on Ag{x), we may choose d G Zg{x)^ . We have 

h-^F^{h) = h-^F{h) ■ F{h-^F{h)) = dF{a) = d^. 

It follows that any clement g G is G^'^^-conjugate to g„2 for some a G ^^(a;). 
Hence we have frig) = 'T{a^) = 1 by our assumption. We have 

m,{fr) = E frig) = \C^\/\G^\ 

as asserted. The lemma is proved. □ 



5.4. Assume that G is as in 3.1, and we use the notation there. Let A = 
IC(G, f )[dimG] be the cuspidal character sheaf on G. (Here £ represents one of 
the simple local systems £,£',... on G if there exist more than one). Then A is 
F-stable. Let p E Aa{g)^ be the irreducible character corresponding to £. Note 
that F acts trivially on Acig)- We consider the class function fp on Gy„i defined 
as in (5.2.1) for t = p. We show that 

(5.4.1) m2(/p) = ^-(dimG-dimC) 

In fact thanks to Lemma 3.2, one can apply Lemma 5.3 and we have m2(/p) = 
|G^|/|G^|. Since F acts trivially on Aa{g), splits into several G^-conjugacy 
classes, which are parametrized by Acig)/^. Let ai, . . . , be the representatives 
of the conjugacy classes in Acig), and let Ci the G-'^-conjugacy classes in G^ cor- 
responding to flj. We choose gi G C,. Since \ZG{gi)^\ = \AA{ai)\\ZQ{gi)^\, where 
A — Aoig), we have 

|C^I/|G^I = E |Ci|/|G^| = E \ZA{ai)nz'G{9if\-\ 

i=l i=l 

Here Z^igi)^ ~ Z'jj,{ui)^ , where g^ = SiUi = UiSi and Hi = ^^.(sj). Note that 
Z'h — ^^'i since w G Go, where (Go, ^o) is a cuspial pair with unipotent 

support, it is known by [L2,I, Prop. 3. 12] that Zjj{u) is a unipotent group. It follows 
that 

\Z%{gi)^\ = \Z%.{Uif\ = gdimH-dimCo ^ ^dimG-dimC_ 
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Hence we have 

\C^\I\G^\ = g-(dimG-dimC) ^ |Z^(a,)|-l = g-(dim G-dixn C) _ 

i=l 

This proves (5.4.1). 

Let ifo : {F'^)*£:^£, (p : {F'^)*AzA and (f>A : iF'^)*A2^A bo as in 3.1, but 
replacing F hy F^. Then the characteristic function xs,ipo '^^ coincides with 
/pI^f2. Since A is clean, the function XA,ip coincides with fp\—p2. It follows that 
Xa coincides with gdimG-dimC^^ r^YAs. imphes that 

Lemma 5.5. Let A he a cuspidal character sheaf of G. Then we have 

m2{XA) = 1- 

5.6. Let s e T* be such that the class {s} is F^-stable. Then there exists 

So 6 {s} such that F^(so) = sq. Let H = Zq{so). Then H is an i^^-stablc reductive 
subgroup of G. It is known, since the center of G is connected, that there exists a 
natural bijection £{G^ £{H^ ){!})) P ^ Puni- Concerning the values of 

m2{p) and TO2(/0uni), the following result is known. (For unipotent characters, we 
follow the notation in Section 4.) 

Theorem 5.7 ([K], [L4]). Let G he a connected classical group with connected 
center. Then 

(i) // there does not exist sq S {s}^^ such that F{sq) = s'^^ , then m2{p) = 
for any p G £{G^ , {s}). If there exists such sq, then under the notation 
of 5.6, m2{p) = m2(/9uni) for any p e £{G^ , {s}). 

(ii) Assume that G is of type Bn or Cn- Let he a family in such that 
\Zi \ = 2rfi + 1 (cf 4.2). Then we have 

if A is special, 

otherwise. 

(iii) Assume that G is of type Dn- Let T be a non- degenerate family in 
such that \Zi\ = 2di (cf. ^.2). Then we have 




m2{pA) = 



2''i ^ if A is special, 
otherwise. 



If = {A, A'} is a degenerate family, then we have 
m2{pA) = m2{pA') 



1 if F is of split type, 
otherwise. 



In view of (4.2.2), we have the following corollary. 

Corollary 5.8 ([L4]). Let G be a classical group of split type. Assume that 
T is a non- degenerate family. Then for any A G J^, we have m2{RA) = 1- 

5.9. Let A be as in 5.4 and assume that A e Gc. Let s e T* be such that 

the class {s} corresponds to C via / in 1.2. Then = 1. Let Rq be the almost 
character of G^ corresponding to A as given in Theorem 1.5. Then by Lemma 5.5, 
we have m2{Ro) ^ 0. Note that Rq is a linear combination of irreducible characters 
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contained in £{G^ , {s}). Thus by Theorem 5.7, (i), there exists so G {s} such that 
So € and that -F'(so) = Sg ^ = so- Then H = Zq{sq) is an F-stable reductive 
subgroup of G, and so H is spht over Fq2 . Recall that the almost character Rq is 
given as in (1.3.1). Then it is known that (-l)'(^i) = a{H)a{G), where a{G) is 
a split rank of G with respect to Fg2, and vice versa for H. Since G,H are split, 
wc have l{zi) = a{H)a{G) = 1. Let Rq be the almost character of H^^ obtained 
from Ro under the correspondence p ^ puni- Then we see that 

(5.9.1) m2(i?o) = ™2(i??) = 1. 

In fact, the first equality follows from Theorem 5.7, (i), together with the fact that 
l{zi) = 1. The second equality follows from Theorem 5.7, (ii), (iii). 

Combining Lemma 5.5 with (5.9.1), we have the following theorem. 

Theorem 5.10. Let G be an adjoint sim,ple group of classical type. Let A he 
a cuspidal character sheaf, and XA = XA,tj>A be the characteristic function of A on 
G^^ (defined as in 3.1). Let Rq be the almost character of G^^ corresponding to 
A. Then we have 

Ro = XA- 

As a corollary we have the following result, which holds without any restriction 
on p nor q. 

Corollary 5.11. Let G be a connected classical group with connected center. 
Then the constants appearing in Lusztig's, conjecture (Theorem 1.5) can be de- 
termined for G^ in the following cases; under the notation of 1.6, assume that 
Wg = Zg. Then we have 

R.. = (-1)^™''XA« 

for any E G W^. In other words, we have Cxe = (—1)'^™'^. 

Proof. Lemma 1.8 together with the argument in 1.9 shows that the deter- 
mination of Cx is reduced to the case of Co (the one corresponding to the cuspidal 
character sheaf) in the case of adjoint simple groups. We know that Co = 1 by 
Theorem 5.10. Then the corollary follows from Lemma 1.8, together with 1.9 since 

lC=l^'. □ 

Remark 5.12. In the case where G = PSp2n or S02n+i, Corollary 5.11 gives 
a complete answer for the determination of constants (x for G since we have 
alwyas Ws = Zs in that case. In the case where G = PS02n, the corollary holds 
if RxE is a linear combination of unipotent characters, i.e., if Ax^ G Gq^. But it 
happens that Ws ^ Zg for some 5. 

6. From G^' to G^ 

6.1. The results Theorem 5.10 and Corollary 5.11 in the previous section 
are only valid for the group G . In this section, by using a certain specialization 
argument, we extend those results to the group G^ as far as xa are concerned with 
unipotent characters. In the case of p = 2, this implies the extension of Theorem 
5.10 and Corollary 5.11 for the group G^ of split type. We have the following 
theorem. 
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Theorem 6.2. Let G be a classical group of split type over Fg. Let A be a 
cuspidal character sheaf contained in Gq^ , and xa = Xa,4>a characteristic 
function of A over G^ . Let Rq be the almost character of G^ corresponding to A. 

Then we have 

Ro = XA- 

As a corollary, we have 

Corollary 6.3. Let G = Sp2n or S02n with p = 2. Assume that G^ is of 

split type. Then the constants (x appearing in Lusztig's conjecture (Theorem 1.5) 
can be determined completely for Sp2n, and partly for S02n- More precisely, under 
the notation of 1.6, assume that Ws = Z^. Then we have 

(6.3.1) i?,, = (-l)<i™^XA« 

for any E e W^. In particular, (6.3.1) holds if Ae € Gq^ in the case where 

G = S02n. 

Proof. As in the proof of Corollary 5.11, the determination of C,x is reduced to 
that of C,Q. Assume that G = Sp2n or S02n with p = 2. In this case, it is known by 
3.1, (c), (d), that a unique cuspidal character sheaf (if it exists) is always contained 
in . Hence Theorem 6.2 can be applied, and the corollary follows from Lemma 
1.8 (see also Remark 5.12). □ 

Remark 6.4. A similar argument as in Corollary 6.3 works also for the case 
where p ^2. In particular the formula (6.3.1) holds for the case where Ae G Gq^. 
Thus we rediscover the results in Theorem 6.2 in [S2]. Although the argument 
given in [S2] can not be applied to the case where p = 2, the proof here works 
simultaneoulsy for arbitrary p. 

6.5. The remainder of this paper is devoted to the proof of Theorem 6.2. In 
what follows, we assume that G is a classical group containing a cuspidal unipotent 
character. Hence G is of type Bn or C„ with n = d^ + d, or of type Dn with 
n = Ad?. We assume further that G^ is of split type. We follow the notation in 
Section 4. 

Let Iq bo the G^-module IndgF 1 induced from to G^ . Then the irreducible 
component of Iq is in bijective correspondence with W^. We denote by the 
irreducible G^-module occurring in Vq corresponding to E G W^. pE gives a 
unipotent character, which we denote by with Ae 'E X„. Let Ti be the Iwahori- 
Hecke algebra over Q[m^/^, associated to the Coxeter system {W,S) with 

generators {Tg | s € S}. H has a basis {T^ | w G W}, where is defined as 
T^u = Tg-^ . . . Tgf^ for a reduced expression w = Si^ ■ ■ ■ Si^. H is characterized by 
the following properties; 

(Ts - u){T, + 1) = 0, 
TsT^ = Tsw if l{sw) = l{w) + 1, 

where I -.W ^ Z>o is the length function of W. 

In the case of type B„ or the generator set S of is described as follows. 
Assume that W is the Weyl group of type i?„. Then W is realized as a group of 
signed permutations of / = {1, 1, 2, 2, . . . , n, n}. The set S of generators is given as 
S = {so, si, . . . , Sn-i} with So = (1, i), si = (1, 2), . . . , s„_i = (n - 1, n), and we 
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denote by Ti the generator of H corresponding to Si . Note that the subalgebra of Ti 
generated by Ti, . . . ,T„_i is isomorphic to the Iwahori-Hecke algebra of type An-i- 
Next assume that W is of tyep Then W is a subgroup of the Weyl group of 
type Bn, generated by Sg = (1,T)(2, 2), si, . . . , s„_i. We denote by T,^, Ti, T„_i 
the corresponding generators of H. 

The endomorphism algebra End^F Iq is isomorphic to the specialized algebra 
Qi ® H via the algebra homomorphism Q[m^/^, ^ Q; by u ^-> q, which we 

denote by Hq. We denote by Eg the irreducible representation of Hq corresponding 
to i!^ e W^. Now Iq has a structure of x Hg-module, and the trace for g e 
, e Tiq is written as 

(6.5.1) Tr((5,r^),/,)= T^^{9,PE)Tv{T^,Eq). 

Eew^ 

By replacing pe = Pab by Ra by using (4.2.3), we have 

(6.5.2) Tr((5,r^),/,)= ^ fAiw)RAig), 

Aex„ 

where 

(6.5.3) fAiw)= J2 {A,AE}Tr{Ty„Eq). 

Eew^ 

It is known that Tr{Tyj,Eq) is a polynomial in q in the sense of 2.3. Hence 
fA{w) is also a polynomial in q. We are interested in fA{w) in the case where A 
is the cuspidal symbol A^, and we want to find some special w £ W such that 
/^c(^) 7^ 0. Let W be the Weyl group of type or Then any element 

of W can be expressed as a product of positive cycles and negative cycles, where 
the number of negative cycles is even if W is of type We have the following 
proposition. 

Proposition 6.6. There exists an element w €W such that Ja, (w) ^ 0, where 
either w is a Coxeter element in W, or w contains a positive cycle of length > 2. 

6.7. The proof of the proposition will be given in Section 7. Here assuming 
the proposition, we continue the proof of the theorem. We prove the theorem by 
induction on the semisimple rank of G, and so wc assume that the theorem holds for 
the classical groups of the smaller semisimple rank. Let A be the cuspidal character 
sheaf on G as in the theorem, and let C be the conjugacy class which is the support 
of A. We choose g = su = us E as in 3.1. We choose w S as in Proposition 
6.6. We consider the equation (6.5.2) simultaneously for the groups G^ for any 
integer m > 1. Note that g G G^ and it has a uniform description for any m > 1 
since the split unipotent element for G^ is split for any extended group G^ . Then 
we can write (6.5.2) as 

(6.7.1) Tr((5,T^), V) = J2 fAHiq^R^rHg) + fAA^v^q^Rt^d), 

A^A, 

where is an element of TLq^ , and denotes the almost character of G^ . 

By induction hypothesis and by Remark 6.4, the formula (6.3.1) holds for i?^-* if 
A^ Ac- We show the following lemma. 
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Lemma 6.8. Assume that A ^ Ac- Then under the induction hypothesis, 
R^^\g) is a rational function in q. 

Proof. Let Aa the character sheaf corresponding to Ra, and denote by 
the characteristic function on G^"^ associated to Aa- In view of (6.3.1), it is 
enough to show that x^a!^ (.9) rational function in q. Since A e Gq^ , by 

applying Corollary 4.10 in [S2] (oq = 1 in the notation there since F is of split 
type), the computation of XA^Ji9) is reduced to that of Xa^J{u) for a subgroup 
if^"". So we consider Xa^J on . is a direct summand of a certain complex 
K = indp Aq, where Aq g Lq^ is a cuspidal character sheaf of a Levi subgroup L. 
Then the computation of Xa^ Ig""" reduced to the computation of the generalized 
Green functions of G^"" associated to L^". Hence by Corollary 2.4, we obtain the 
lemma. □ 



6.9. Next wc consider the left hand side of (6.7.1). The G^-module Iq is a 
permutation representation of G^ on / . Since the action of G^ on G^ /B^ 
is independent of the isogeny, we may assume that G = SpN or 50 jv and that 
g G G^ . Let V be the vector space over of dimT^ = N, equipped with a non- 
degenerated alternating form (resp. symmetric bilinear form) / on if G = Sp^ 
(resp. 50jv). In the case where G = SOn with p = 2 and N is even, we also 
consider the quadratic form Q on V. Now the set G^ / B^ may be identified with 
the set of flags J^q as follows; A flag F = (Vq C Vi C • • • C V^) is a sequence of 
subspaces of V such that dimV^ = i and that Vi are isotropic with respect to /, 
where TV = 2n, 2n+l or 2n according to the cases where G = Sp2n, S02n+i or S02n- 
In the case where p = 2 and G = S02n, we assume further that the restriction of 
Q on is zero. Now in the case where G = Sp2n or S02n+i, consists of all 
flags on V . G^ acts naturally on J^q via x : (Vq C • • • C Vn) ^ (xVq C • • • C xVn) 
for X G G^ , and the G-'^-set J^q is identified with the G-'^-set G^ / B^ . In the case 
where G = S02n, we define ^q as the set of all flags on V as above, then G^ 
acts on Tq withtwo G-'^-orbits, Tq and J-'q. Either of them can be identified with 
qP I ^ and wc have a natural bijection between J-q and J-q, which is given in the 
form (Vb C Vi C • • • C K) (Vb C Vi C • • • C V;'), (only the term Vn is changed 
to K). 

We consider the vector space Jq over Qj with basis JF^, which is identified with 
Iq. By the identification Iq Jq, Hq acts on J7g, whose action is given as follows; 
let F = (Vb C • • • C Vn). For i = 1, . . . , n — 1, we have 

FTn-i = ^ (Vb C • • • C C C V-+1 C • • • C Vn), 

where the sum is taken over all the isotopic subspaces W such that Vi-i <zWc Vi+i 
and that dimT4^ = W ^ Vt. In the case of B„, we have 



FTo= {VoC--- cVn-iCW), 

WDV„-i 
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where the sum is taken over all the isotropic subspaces W such that diniM^ = n 
and W ^Vn- In the case of type Z)„, we have 

FT^= Yl {VoC---C Vn-2 cWc W), 

where the sum is taken over the isotropic subspaces W D W such that dimW = 
n, dim W = n — 1 and W contains Vn-2 and some more conditions. 
It follows from the description of the action of H on Xg, we see that 

(6.9.1) Assume that w E W contains a positive cycle of length > 2. Then there 
exists A: > 1 such that for any F = (Vq C • • • C Vn), FTyj is a linear combination of 
F' = (Fo' C • • • C such that Vk = V^. 

We show the following lemma. 

Lemma 6.10. Under the induction hypothesis, Tr {{g,Tyj),Iqm) is a rational 
function in q. 

Proof. The following argument was inspired by [HR2], where the combinato- 
rial properties of Tr ((m, T^), Iq) is discussed in the case of GL„(Fq) with a uiiipo- 
tent element u. First assume that w contains a positive cycle of length > 2. Then 
by (6.9.1), for any flag F = (Vg C ••• C Vn), FT^ is a linear combination of 
F' = {Vq C • • • C Vn) such that Vk = V^. We now prepare a notation. If t; e Jq, 
and F € Tq, we denote by z;|f the coefficient of F in the expression of w as a linear 
combination of base vectors. Let F = (Vq C • • • C Vn) E Tq, and assume that 
5FT^|f 7^ 0. Since ^FT^If = FT^j^-iF, g^^F = {V^ C ■ ■ ■ C V^) is of the form 
that Vk = Vl = g~^Vk- It follows that Vk is stabilized by g. Thus we have 

Tr((5,T^),7g)= ^ 5FT^|f 

w F=(yoc---cwc---cv„) 
= E E 5FX|f' 5F'X|f" 

W F'=(VbC---CVF) F"=(W<Z-) 

where W runs over all the isotropic subspaces in V such that dim W = k and that 
gW = W. Let H' = GL{W), and H" be the group of isometrics Sp(y7) or S'O(W) 
for W = W-^/W. Let ~ be the corresponding induced modules for H'^ , 
and similarly define ~ J'^ for H"^ . g acts naturally on W (resp. on W), and 
we denote by gw E H'^ (resp. g^^ G H"^) the corresponding elements. Also the 
action of on Iq induces an action on (resp. on J^) which is given by T^,/ 
(resp. Tw") with an element w' (resp. w") in the Weyl group of H' (resp. H"). 
Then the last sum can be written as 

(6.10.1) Tr {{g, T^), /,) = Tr {{gw, T^'), /f) TV {{g^, T^.), -^f )• 

w 

Let Fq = {Vq C • • • C 1^) be the standard flag whose stabilizer in is , 
and put Wfc = V°. Then there exists an F-stable maximal parabolic subgroup 
P of G containing B such that is the stabilizer of Wk in . Let L be an 
P-stable Levi subgroup of P containing T. Then L is isomorphic to L' x L" , where 
L' = GLk and L" is a similar group as G of rank n — k. Let g[,...,g'r (resp. 
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§'{,..., g'g) be representatives of the conjugacy classes in L'^ (resp. L"^) such that 
Qij = {gi,g'j) S is conjugate to g under . We denote by X'^^ the set of 
W such that W = xWk and that x~^gx is conjugate to gij in . Then (6.10.1) 
implies that 

r s 

(6.10.2) Tr {{g, T^), h) =Y.Y. T^'), I^') Tr {{g'' , T^"), /f ), 

where ,1^ are corresponding induced modules for L'^ ,L"^ . Choose Xij € G^ 
such that g = XijgijX^j^ for Then X^^ is in bijection with the set Zcig)^ XijL^ / , 
hence \Xi^\ = \ZG{gr\/\ZL{gijr\. 

We now consider (6.10.2) for any G^ . Then the choice of representatives 
gij e G^" does not depend on m, and we see that \Xg-L \ is a rational function in 
q. On the other hand, one can write as in (6.5.1) 

Tr {{g'J, T^„), jf ) = ^ Tr {g'; , pe) Tr {T^.,E^), 

where Wl" is the Weyl group of L" . By induction hypothesis, i?^' (g") is a rational 
function in q for any almost character Ra of L"^ . Hence Tr (</", p^^) is a rational 
function in q. It follows that Tr {{g'j, T^"), Iqm) is a rational function in q. Similarly, 
and as it is known since L' — GLk, Tr ((g^, T^/), /^™) is a rational function in q. 
Thus we conclude that Tr ((^i, T^), Jgm) is a rational function in q as asserted. 
Next assume that w; is a Coxeter elment of W. We note that 



(6.10.3) Tr((x,T^),/g) = 



q^ if Xu is regular unipotent, 
otherwise, 



for X G G^ , where Xu is the unipotent part of x and r is the semisimple rank of 
G. In fact (6.10.3) is discussed in [IIR2, Prop. 3.2] in the case where G = GLn- 
The argument there works in general if we notice that Zg{v) = Zij{v) for a regular 
unipotent element v G and that \Z^{v)\ = q^, where U is the unipotent radical 
of B. (6.10.3) implies that Tr {{g, T^), Iqm ) is a polynomial in q. Hence the lemma 
holds. □ 



6.11 We now prove the theorem. In the formula (6.7.1) the lef hand side is 

a rational function in q by Lemma 6.10. For for /I 7^ ^, i?^"Hg) is a rational 
function in q by Lemma 6.8. Since fA^w) is a polynomial in q, and //i^(w) 7^ 
by Proposition 6.6, we see that R^y^\g) is a rational function in q. By Theorem 
1.5, one can write R^^\g) = C^™''Xa"^(5) ^^^^ some C^™) G Q;* of absolute value 
1, where ^0 = -^a^ is the cuspidal character sheaf. We know that X^A^id) ^ 
non-zero polynomial in q. We also know by Theorem 5.10 that C^"*^ = 1 for even 
m. It follows that R^A^\,q) / X'a^^ [g) is a rational function in q, and takes the value 
1 for any power of q^ . Hence R'^\g) = X'Al\g) for any m, and we have C*-™'' = 1- 
This shows that Ra^ = XAq, and the theorem is proved (modulo Proposition 6.6). 
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7. Proof of Proposition 6.6 



7.1. Recall that is the set of symbols of rank n and defect 1 as in 4.1. In the 
case where W is the Weyl group of type S„, the set is in bijection with The 
correspondence is given as follows; let Vn be the set of of double partitions (A, ^) 
such that |A| + = n. Then is parametrized by Vn- For a double partition 
(A,/i) e Vn, we write A = (Aq < Ai < • • • < A^) and n = {^i < H2 < • - • < IJ-m) 
with Ai, /Ui > for some integer m. We put tti = Xi+i, hi = and define the 

sets S,T hy S = {ao, ai, . . . , a^} and T = {6i, . . . , Then yl = (^) e c()^, and 
this gives a bijective correspondence between and Vn, and so gives a bijection 
between and W'^. As in 6.5, we denote by yl^; the symbol in corresponding 
to £■ € W^. Assume that n = (P + d, and let Aq be the special symbol in Then 
Ao = Ae, where E corresponds to (A, n) G Vn such that A = (0<l<2<---<d), 
^ = (1 < 2 < • • •^< d). 

Recall that $° is the set of symbols of rank n and defect as in 4.1. In the case 
where W is the Weyl group of type D„, is in bijection with which is given 
as follows; let Vn be the set of unordered partitions (A, /x) such that |A| + = n, 
where (A, A) is counted twice. Then is parametrized by Vn- For (A,/i) 6 Vn, 
we write A = (Ai < • • • < Xm), A* = (a*i < • • • < Mm) with Ai,/ij > for some 
integer m > 1. We put ai = A, + (i — 1), bi = fii + {i — 1), and define the sets 

S.T by 5 = {fli a,,,}. T = . . .,&,„}. Then yl = (^) e $o and this gives 

a bijective correspondence between $° and W^. As in 6.5, we denote by Ae the 
symbol in $° corresponding to £^ € M^^. Assume that n = A(P and let Aq be the 
special symbol in Tc- Then /lo = /Ib, where corresponds to (A, /z) G such 
that A = (0 < 1 < • • • < 2q! - 1). /i = (1 < 2 < • • • < 2d) 

First we show the following lemma. 



Lemma 7.2. Assume that E G is such that Ae G Tc- If E corresponds to 
(A, /x) G Vn (resp. (A,/u) G 7^„j, m i/ie case where W is of type Bn (resp. Dn), 
then we have 



{Ac,Ae} = < 



2^ 



(_l)|A|+d(d+i)/2 Q ^yp^ 



L22rf 



Proof. In the case where T ~ !Fc is the cuspidal family, Zi = {0,1,..., 2d} 
(resp. Zi = {0,1,.. .,4d-l})andMo = {1, 3, . . . , 2d-l} (resp. Mq = {1,3,.. .,4d- 
1}) if G is of type S„ (resp. D„) by 4.3. Moreover A^ = Am with M = II). Then 
Ml* = Afo, and for any M' C such that \M'\ = d (resp. |M'| = 2d), we 
have M'^ n Af» = Afo - Af. Now Af is written as Af = M^^^UAf^^, where 
-^odd (resp. Af^v) is the subset of M' consisting of odd numbers (resp. even num- 
bers). We have A'/' n A'/q = Af^dd ^'^o - A'/' = Mq - Af^^^. Moreover we have 
Zi-M' = ((Zi)ev-Af^^) 11(^0- Afodd)> where (Zi)ev is defined similarly. Assume 
that M' corresponds to (A, fj,) G Vn (resp. (A, fi) G 'Pn). Let 7 = Ylf^i i = d{d+l) /2 
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(resp. 7 = Y.Zi^ ^ = d{2d - 1)). Then 

|A| = ( E «)-^ 

aeZi-M' 

= |Mo-M^dd|-7 (mod 2). 
This proves the lemma. □ 



7.3. Wc consider //i^(w) G Zlu^^"^ ,u~^^'^] for some w G VK, and compute it 
by making use of the Murnaghan-Nakayama formula for H due to Halverson and 
Ram [HRl]. We use the notation in 6.5. 

First assume that W is of type i?„ and H is the corresponding Heckc al- 
gebra. For each 1 < A: < n, we define Lk £ H inductively as Li = To and 
Lk = Tk-iLk-iTk-i for = 2, . . . , n. For 1 < < Z < n, we define 

Rki = TkTk+i ■ ■ ■ Ti-i, Rki = LkTkTk+i ■ ■ ■ T;_i. 

For an element i or i in /, we put \i\ = \i\ = i. For a sequence r = (ri, . . . ,rfe) of 
elements of / such that |ri| < |r2| < • • • < |rfe|, we define Tr e W by 

Tr = _Ri_ri-R|ri|+l,r2 ' ' ' ^|rfc-l| + l,rfc ■ 

Then Tr coincides with T^^, where G W is given by a cyclic notation of the 

signed permutation, 

g Wr =(1, 2, . . . , Inl - 1, ri)(|ri| + 1, Inl + 2, . . . , jraj - 1, rz) 

•••(|rfe-i| + l,...,|rfe| -l,rfe). 
We also use the following cycle type expression of Wr 

(7.3.2) Wr = [h,...,lrl 

where k G I is such that \li\ = \ri\ — and k is barred if is barred. For 

example, if r = (1,4,7, 12), then Wr = [1,3,3,5]. 

We now prepare some notation related to the skew diagram. Let A be a double 
partition of size n. Apart from the notation in 7.1, we express it as A = (A", A^), 
where A", A'' are partitions. For fj, &Vn, we write /x C A if /i" C A" and fj,^ C X^^ . 
The Young diagram of A is defined as a pair of Young diagrams of A" and A'^. 
We often identify the double partition and the corresponding Young diagram. For 
double partitions C A, the skew diagram X/n = ((A//Li)", (A//^)'^) is defined 
naturally. For each node x in the skew diagram A//i, the content ct{x) is defined 
as follows; 



(7.3.3) 




if a; is in position (i, j) in (A//i)", 
—u^~^ if X is in position in (A//i)^. 



The skew diagram X is called a border strip if it is connected and docs not 
contain any 2x2 block of nodes ( "connected" means that two nodes are connected 
horizontally or vertically). The skew diagram X is called a broken border strip if 
its connected components are border strip. Note that a double partition (a, /?) with 
both a, /3 non-empty consists of two connected components. For a border strip X, 
a sharp corner is a node with no node above it and no node to its left. A dull corner 
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in a border strip is a node which has a node to its left and a node above it (and so 
has no node directly northwest of it). 

For a skew diagram X, let C be the set of connected components of X, and 
put m = |C|, the number of connected components of X. We define A{X), A{X) e 
Z[uV2^u-i/2] as follows; 



A{X) 



'(^1/2 _ «-/2) — 1 JJ (^l/2)c(y)-l(_^-l/2)r(y)-l 

if X is a broken border strip, 

otherwise. 



A{X) = I 



yeDC zeSC 

if X is a (connected) border strip, 

otherwise, 



where SC and DC denote the set of sharp corners and dull corners in a border 
strip, and r{X) (resp. c{X)) is the number of rows (resp. columns) in the border 
strip X. 

The Murnaghan-Nakayam formula for H. by Halverson-Ram is given as follows. 
Note that in the formula below, l'{w) denotes the number of Si, . . . , s„_i (excluding 
So) occurring in the reduced expression oi w £W 

Theorem 7.4 ([HRl, Theorem 2.20]). Assume that W is of type Bn- Let 
he the irreducible representation ofH associated to \&Vn- Then 

Tr(r^„i;„^) =t.''(-)/2 ^ Z\(M«)Z\(M(^)/MW)...Zi(/zW/M('=-i)), 

0=/n((')C^(i)C...C/:i(fc)=A 

where the sum is taken over all the sequences = /x'^^ C C • • • C jjS^^ = A such 
that I/x'^^Va*^*^""^^! = kfcl ~ and the factor A{^i^^^ / is barred ifru in r 

is barred. 



7.5. Next assume that W is the Wcyl group of type £>„ and Ti is the 
corresponding Hecke algebra. Then under the notation of 6.5, we define L'f. by 
L[ = 1,L'2 = T^Ti, and L'^. = Tfe_iL'^_;^Tfe_i for A; = 3, . . . , n. For 1 < fc < Z < n, 
we define 

Rki = TkTk+i ■ ■ ■ T(_i, R/^i = L'fJTkTk+i ■ ■ ■ T/_i. 

For a sequence r = (n, . . . , r^) of elements / such that |ri| < |r2| < • • • < |rfe| and 
that the even numbers of r, are barred, we define Tr e W by 

Tt = -Rl,ri-R|ri|+l,r2 " ' " ^\rk-i\+l,rk- 

Note that Tr does not always correspond to for some w, but it corresponds to 
Tyj^ for Wr GW as given in (7.3.1) in the following two cases, 

(i) > for i = 1, . . . , k, 

(ii) ri = —1, r2 < and rj > for i = 3, . . . , fc. 

In what follows, we only consider r as above, and so assume that = T^^- 
Note that Wr is regarded as an element of the Weyl group of type Bn in the notation 
of 7.3. 
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As in 7.3, we consider the skew diagram A//x, and define ct'{x) by modifying 
(7.3.2), 




* if a; is in position in (X/fi)", 

—u^~^ ii X is in position in {X/ fj)^ . 



For a skew diagram X, we define A{X),A {X) G Z[ii^/^,u ^Z"^] by the formula in 
7.3, but for A (X), we modify the definition of A{X) by replacing ct(x) by ct'{x). 
Concerning the irreducible characters of W, we have the following result. 

Theorem 7.6 ([HRl, Theorem 4.21]). Assume that W is of type D„. Let 
be the irreducible representation of Ti associated to \ & Vn such that A" 7^ . 
Then Tr iT^^,E^) can be computed by the formula in Theorem 7.4 for type Bn, by 
replacing A{X) by A{X). 

7.7. Let Tc is the cuspidal family as in 4.3, where Tc is a subset of or <!>+. 
Let J^l (resp. J^) be the set of symbols of defect 1 (resp. defect 0) contained in 
Tc- First assume that C Let be the set of double partitions A such that 

€ J^c- Then A G can be written as A = (a,/?) with a : ai > Q!2 > • • • > 
ctd > aa+i > 0, and /? : /3i > /32 > • • • > /9d > 0. Let /?* : /3J > > • • • be the 
dual partition of /3. The following fact is easily checked. 

(7.7.1) Tl c^vi = {X= {a, /?) e K | + P*a-i+2 = d ior 1 < i < d + 1}. 

Next assume that !Fc C Let T"^ be the set of double partitions X GVn such 

that Ae>- & Tc-, where A = (a, /?) with a : ai > • • • > > 0, /? : /3i > • • • > /3d > 
(in this case always a ^ (3). Let /?* be the dual partition of /?. Then we have 

(7.7.2) c^Vt = {X= [a, 13) e Pn \ a, + = 2d for 1 < i < d}. 

(7.7.1) shows that A = (a,/3) G 2 is obtained from a diagram 7 = (d'^+^) of 
rectangular shape as follows; take any partition a C 7, and let (3 be the dual of the 
partition obtained by rearranging the skew diagram 7/0:. Similarly, (7.7.2) shows 
that A = {a, (3) G is obtained from the diagram 7 = ((2(i)^'^) of rectangular 
shape by a similar process as above. 

For example, in the case of type B„ with d = 2, we have n = d{d + 1) = 6 and 

Vi = {(21; 21), (2^; l2), (l2; 31), (23; _), 1), 
(2l2;2),(l3;3),(2;22),(l;32),(-;32)}. 
In the case of type Z)„ with d = 1, we have n = 4d^ = 4 and 

Pi = {(22;-),(2l2;l),(2,l2)}. 

7.8. Let Wr be an element in W associated to some r = (ri, . . . ,rfc) as in 
(7.3.1). By Lemma 7.2 and by (6.5.3), we have 

(7.8.1) fA^y^r) = S ^ (-l)l«l TV (T^,, £;(«''')), 

where the sum is taken over all A = (a,/?) in (resp. in Vf^), and the constant S 
is given as 6 = {-l)did+i)/22-d (rggp, (_;L)d(2d-i)2-2d+i) jf q jg ^ypg (j.ggp_ 
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Wc shall compute this sum for some specific choice of r by applying the 
Murnaglian-Nakayama formula (Theorem 7.4 or Theorem 7.6). In order to dis- 
cuss the case Bn and £)„ simultaneously, we consider the following setting. Let 
V^''' be the set of double partitions A = (a, /3) G Vn, where a : ai > • • • > > 
0, /3 : /?! > • • • > > and n = ab, such that 

(7.8.2) P"'*- = {A = (a, l3)GVn\ai+ = 6 for 1 < i < a}. 

Hence V^'^ is the set of A contained in the Young diagram 7 — (&°) of rectangular 
shape in the above sense. In particular, V^'"^^^ coincides with V^, and V^'^'^ (under 
the identification (a,/3) = (/3, a)) coincides with V!^. 
Put 

(7.8.3) fa,b{wr)= E (-l)l"lTV(T^,,i;(«''5)). 

(a./3)eK''' 

We take = n,rk-i = n — (2a + 2& — 6) so that in applying Theorem 7.4 or 
Theorem 7.6, ii^^'> / is a broken border strip of length 2a + 26 — 6. Let X be 
a broken border strip of length 2a + 26 — 6 contained in A = (a, 0) G Vf{^ . We can 
write X = Y IJZ with F C a, Z C /3, broken border strips. Since the maximum 
length of a border strip is a + 6 — 1, we have only to consider the following 5 cases. 

CaseL |r| = a + 6 - 1, |Z| = a + 6 - 5, 
CaselL |y| = a + 6 - 2, |Z| = a + 6 - 4, 
Casein. \Y\=a + b-3,\Z\ = a + b-3, 
Case IV. |y| = a + 6 - 4, |Z| = a + 6 - 2, 
Case V. \Y\=a + b-5,\Z\=a + b- 1. 

We consider the diagram 7 = (6") of rectangular shape so that a U /3* = 7. In 
the following discussion, we regard Y and Z as paths in 7, instead of considering 
a and P separately. For example, the following figure explains an example of the 
case I, where A = (a, /3) = (42^12, 4^21) with n = 20, F is a border strip of length 
8 and Z is a border strip of length 4. In the figure, • (resp. x ) denotes the starting 
point and the ending point of Y (resp. Z). 
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7 = (45) a = (42^12) /3 = (422) 

In computing fa,b{wr), we use the following cancellation property. 

Lemma 7.9. Let x he the top rightmost node ofj, and y the west of x, z the 
south of X. Assume that x,y €Y and that z G Z. Then a : ai > a2 > ■ ■ • , P : Pi > 
/?2 > • • • with a\ = a,0i =6—1. Let X' = {a', /?') G Vn be defined by a[ = ai — 1, 
P[= P^ + l and a'j = 0.^,0] = /3j for j ^ 1. Then A' G V^^'' . Put Y' ^Y - {x\ 
and Z' = Z + {x}. Then Y' (resp. Z' ) is a broken border strip of a' (resp. P'). 
Let X' = Y'\[Z'. Then we have 

A{X) = A{X'). 
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Moreover, the double partition {a — Y, (3 — Z) coincides with (a' — Y',13 — Z'). 
Hence in the computation of fa,biwr), the broken border strip X of this type may be 
ignored. Similar situations occur also for the cases, such as x is the bottom leftmost 
node of and y is the north of x, z is the east of x, and x,y €Y, z G Z. 

Proof. Since y €Y and z E Z, the number of border strips in X is the same 
as the numbers in X'. Since c(Y') = c{Y) - l,c{Z') = c{Z) + 1, and r(Y') = 
r(F), r{Z') = r{Z), we sec that A{X) = A{X'). Clearly, we have {a-Y,l3^Z) = 
(a' - Y',13' - Z'). It follows that in the sequence C /x^^^ C • • • C /z^*^) = A in 
Theorem 7.4 or Theoreni 7.6, C • • • C i-V-^ is comnion for A' ifAtW/M^*^-^^ =X. 
Since \a'\ = \a\ — 1, two terms starting from X and from X' are canceled in the 
computation of fa,biwr) by using these theorems. □ 



7.10. We shall classify the broken border strip X = YY[Z which is needed 

for the computation of fa.b{wr)- Let xt be the top rightmost node and x^ be the 
bottom leftmost node of the Young diagram a. Let yt be the top rightmost node 
of (3 and yb be the bottom leftmost node of /?. Note that we embed /? in 7 by using 
(3* , and regard yi, yi, as a box in 7. 

First we consider the Case 1. We have ai = b,aa 7^ 0, and Xt is the top 
rightmost node and Xb is the bottom leftmost node of 7. F is a unique border strip 
connecting Xt and Xb- Assimie that a2 < b and = 1. Then yt is the south of Xt- 
If Z contains yt or yb, then Lemma 7.9 can be applied, and we can ignore this X. 
Hence Z does not contain yt,yb- In this case, Z is & unique border strip connecting 
the node west of yt and the north of yb- Next assume that a2 = b and Ua = 1- If 
yb G Z, then Lemma 7.9 can be applied. So we may assume that yb ^ Z. In this 
case Z is a unique border strip connecting the node north of yb and yt which is the 
two node south of xt- Next assume that 0,2 < b and = 2. Then yt is the south 
of Xt, and if yt G Z, the lemma can be applied. So we may assume that yt ^ Z, and 
Z is a unique border strip connecting the node west of yt and yb- Finally, assume 
that a2 = b and = 2. In this case, Z is a unique border strip connecting yt and 
Vb- 

Thus Case I is divided into 4 classes, and in each case, Y and Z are determined 

uniquely by A = (a,/?). In a similar way, one can classify all the possible broken 
border strips X for Case II and Case III. Case IV is symmetric to Case II, and each 
class is obtained from the class in Case II, by rotating the diagram 7 by the angle 
180°, and then replacing Y and Z. Similarly, Case V is obtained from Case I. We 
shall list up all the possible cases for the cases I, II, III, in the list below, assuming 
that a, 6 > 4. Case I is divided into 4 classes, Case II into 6 classes, and Case III 
into 12 classes, (Case IV : 6 classes. Case V: 4 classes). Here • (resp. x) denotes 
the starting node and the ending node of the border strip Y (resp. Z). In each 
case, Y and Z are determined uniquely by A = (a, /3). Or alternately, if we draw in 
the diagram 7 the path connecting nodes marked by •, so that it is compatible with 
the path connecting boxes marked by x, then it determines A = (a,/3) uniquely. 

Note that in each case, A{X) does not depend on A belonging to the class, 
and has a common value. We have listed those A{X) for each class, where U = 
(m^/^ — u^^/^). For Case IV or Case V, A{X) is obtained from the corresponding 
A{X) for Case II or Case I, by replacing u^/^ <-> —u~^^^. 
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Case I. \Y\^o, + b-l,\Z\=a + b-5. 



(1) -U (2) u-^U (3) uU 

Case II. \Y\=a + b-2,\Z\=a + b-4. 



(5) -«-l/2[/2 (6) wl/2C/2 

Case III. |F| = + 6-3, |Z| =a + 6-3. 



(4) -U 







• 






X 


• 


x| 




X 







(1) U^/^U^ (2) -«l/2[72 (3) yl/2f/2 _^,-l/2;72 



(1) C/3 (2) -C/ (3) C/3 (4) uU 
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(5) C/3 (6) u-^U (7) -U (8) 



(9) -U (10) u-^U (11) uU 



(12) -U 



7.11. In each case listed in 7.10, one obtains a unique A' G Vn' {n' = 
n — (2a + 26 — 6)) from A by removing the broken border strip X ^ Y Y[Z , where 
A' = {a', (3') with a' = a — Y, (3' = (3 — Z . Let J be the set of classes in the list in 
7.10, and let Qj be the set of A' € Pn' satisfying the condition in each case j. For 
example we consider the case j = I-(l). Then Q = Q/-(i) is the set of A' = (a', /?') 
satisfying the following conditions. 

a' = (a'i,a^,...,a;_3), 
< /3'* = {/3'l,/3%...,l3'l_,) with /3'l = b-l,/3'l_, = l, 
,Q!i +/3'a-i-i = 6 - 2 for i = 1,. ..,a - 3. 

For each A' = (a', /?') € Q, we consider a broken border strip X' of length 2a + 26 — 
10. Then X' is unique, and is given as X' = y ]J Z' , where Y' is a unique border 
strip in a' of length a + 6 — 7, and Z' is a unique border strip in /?' of length a + 6 — 3. 
By removing X' from A', one obtains A" € Vn", where n" = n' - (2a + 26 — 10) = 
(a — 4) (6 — 4). Then it is easy to check that 

(7.11.1) {A" e Vn" I A' e Q} = 

and one recovers the original set V^^'^'^^'^ by replacing a, 6 by a — 4, 6 — 4. We can 
compute that ^{X') = —{v}/^ — it~^/^), which is independent from A' G Q. For 
example, let a = 7,6 = 6 with n = 42. Take A = (64322l2, 6^542) G Vl2^. Then 
A' = (a', /?') = (321^, 6431-^), and under an appropriate rearrangement, 7' = a'U/3'* 
can be drawn as in the following figure. Here Y' (resp. Z') is a unique border 
strip of length 6, (resp. length 10), and • (resp. x ) denotes the starting point 
and the ending point of Y' (resp. Z'). From this, we obtain 7" = (2^), and 
A" = {a",P") = (1,32) gP|'^ 
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X 








• 


























• 








X 











□ 



7' = a' U 13'* 



7" = a" U a" = (1) = (32) 



In fact, a similar kind of arguments work for all other cases, and the set Qj 
is described in a similar way. In particular, 7' = a' U /?'* is of the shape obtained 
from a rectangle by attaching two nodes, one on the above or right of the northeast 
corner, and the other on the below or left of the southwest corner of the rectangle. 
In all the cases, the broken border strip X' = F' ]J Z' of length 2a + 26 — 10 is 
determined uniquely, and we always find the set "P",, ~ after removing the border 
strips X' . Moreover, A{X') takes the common value —{v}^''^ — u~^^^) for all the 
cases through Case I ~ Case V. 

Recall that Wr = {ri, . . . ,rk) with = n. Assume that 

Tk - Tk-i = 2a + 26 - 6, rfe_i - rk^2 = 2a + 26 - 10. 

Thus rfe_2 = n" with n" = (a — 4) (6 — 4). We put r" = (ri, . . . , rk-2) and consider 
Wr" € Wn'i ■ By investigating the above list, we have the following lemma. 

Lemma 7.12. Under the notation above, there exists a non-zero (Laurent) poly- 
nomial h{u) such that 

fa,b{Wr) = h{u)fa-4,b-4{Wr"). 

Proof. For each j e J, we denote by Aj{u) the Laurent polynomial A{X) 
attached to the broken border strip X for Qj as in the list. We put ej = l)'"' 
for A = (a,/3) € Qj. (Note that ej is independent of the choice of A G Qj). By 
(7.11.1), the cardinality of Qj coincides with the cardinality of 'P^/T*'''"*, hence is 
independent of j G J. Then the investigation in 7.11 shows that 

(7.12.1) fa,b{Wr) = Uu'/^ - «-^/^)|P:,7'''"'| ^£,A,(u)|^_4,6-4(^«^")• 

'■ 3&J ^ 

Thus in order to show the lemma, it is enough to sec that ejAj{u) 7^ 0. For this 
we compare the highest degree term u^^'^ in Aj. It follows from the list in 7.10, Aj 
contains the term u^/^ in the following cases, where the coefficients are always 1. 

Case I. (3), 

Case IL (1), (3), (6), 

Case IIL (1), (3), (4), (5), (8), (11), 

Case IV. (2), (4), (5), 

Case V. (2), 

where the numbering in Case IV and V is given by the bijective correspondence 
with Case II and Case I through the rotation of 7. Note that Sj takes the constant 
value for each case, I ~ V. They have the common value for Case I, III, or V, and 
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have a different common value for Case II or IV. This shows that the coefficient of 
u^/^ in SjAj ^ 0. Hence h{u) ^ as asserted. □ 

Returning to the original setting, we show the following two propositions, which 
give the proof of Proposition 6.6. 

Proposition 7.13. Assume that W is of type Bn with n — (P + d. We define 
an element Wr by the cycle type expression given in (7.3.2) as follows. 

[2,12, 16, 12 +16, 16 +16,..., 12 + 16A:, 16+ 16A;], ifd=l (mod 4), 

[6,16,20,16+ 16,20+ 16,...,16+ 16fc,20+16fc], ifd = 2 (mod 4), 

[4,8,4+16,8+16,. ..,4+16A;,8 + 16fc], ifd = 3 (mod 4), 

[8,12,8+16,12 + 16,...,8 + 16fc,12+16fc], ifd = (mod 4), 

for some k > I, where the last term is equal to Ad — 4, and the next term is equal 
to Ad — 8, and so on. Then we have fA^iwr) 7^ 0. 

Proof. We apply Lemma 7.12 with a = d + l,b = d. Then fA^iwr) = 
h{u)fA^{wT") for some non-zero h{u), where r = (n, . . . , rfc) and r" = (ri, . . . , rk-2) 
withrfc = n = d{d+l), rk-r^-i = 4d-4, rk^i-rk-2 = 4(i-8, rfe_2 = {d~4){d-?,). 
Thus the computation of /yi„(wr) is reduced to the case where d= 1, 2, 3. Assume 
that d = 1, then n = 2. One can check by using the formula for A{X) that 
JaAw) 7^ for w = (2). (Note that fAM>) = for ^ = (2))- Next assume that 
d = 2, then n = 6. The direct computation shows that fAa(w) foi' ''^ = (6)- 
Finally assume that d = 3, then n = 12. By using a similar method as in the proof 
of Lemma 7.12, one can show that fA^ (w) ^ for w — (4, 8) (we obtain a similar 
list, but some classes in the list in 7.10 don't appear for this case). This proves the 
pros position. □ 

Proposition 7.14. Assume that W is of type Dn with n = 4rf^. We define an 
element Wr G W as follows. 

[1,3,14,18,14+16, 18+16, ...,14+ 16fc, 18 + 16A:], ifd=l (mod 2), 

[6,10,6+16, 10+16, ...,6 + 16fc, 10+ 16fc], ifd = (mod 2), 

for some k > 1, where the last term is equal to Ad — 6 and the next term is equal to 
M — 10, and so on. Then we have //i^(wr) 7^ 0. 

Proof. We apply Lemma 7.12 with a = b = 2d. Note that since a 7^ /3 for 
any A = (a, f3) G Vn'^''^'^, the argument for type _B„ can be applied without change. 
Then fA^{wr) = h(u)fA^{wr") for some non-zero h{u), whore r = (ri, . . . ,rfe) and 
r" = (n, . . . , rk-2) with rk=n = Ad'^, rt - rt-i = Ad-V>, ru-i - rk-2 = Ad - 10, 
rk-2 = A{d — 2)^. Thus the computation of //i,, (wr) is reduced to the case where 
d = 1,2. Assume that d = 2. Then n = 16. Since a = b = A, Lemma 7.12 can 
be applied, and we see that fAc{w) ^ Q iov w = (6, 10). Next assume that d = 1. 
Then n = A. One can check by using A (X) that fAc{w) 7^ for w = (1,3). This 
proves the proposition. □ 

Remark 7.15. As the formula (7.12.1) shows, our element fAciwr) turns out 
to be if u I— > 1. So our computation cannot be performed in the level of Weyl 
groups. On the other hand, Lusztig showed in [L5] that there exists an element 
w gW such that /yi„('U')|„=i ^ 0. Thus fA^{w) 7^ as a polynomial. This w has a 
simpler form than ours, but since it is a product of negative cycles of the full length 
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(i.e., the sum of the lengths of negative cycles is equal to n), it is not appropriate 
for the computation of the bitrace on the flags (cf. Lemma 6.10). 
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